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Questions  -  Data  Analysis 
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1.  Does  a  single  ISM  sample 
provide  a  reasonable 
estimate  of  the  mean? 

Section  4.2.1 


2.  Can  a  95UCL  be  calculated 
with  ISM  data? 

Section  4.2.2 
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95UCL  =  95%  Upper  Confidence  Limit  of  the  mean 
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Questions  -  Sampling  Design 
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r 

T 

3.  What  sampling  design 
should  1  use? 

Section  4.3.4.2 

L 

4.  Is  it  reasonable  to 
assume  that 
concentrations  are 
similar  across  DUs? 


Section  4.4.2 
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Questions  -  Comparison  of  Data 
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5.  Can  background  and  site 
data  be  compared  using 
ISM? 

Sections  4.4.3.3  and  7.2.4  ■■h 


6.  Can  ISM  data  be  compared  to 
discrete  or  composite  data? 
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95UCL  =  95%  Upper  Confidence  Limit  of  the  mean 


1.  Does  a  single  ISM  provide  a 
reasonable  estimate  of  the  mean? 


*  INTERSTATE  * 
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Answer: 

It  depends  how  much  error  we  are  willing  to  accept. 


►  Why  would  someone  collect  just  1  ISM? 

•  UCL  not  required  by  regulator 

•  Save  time  and  expense 

•  Assumption  that  more  samples 
wouldn’t  change  the  decision. 

For  example 

■  Variance  among  individual  increments  is  low 

■  Mean  of  DU  is  far  above  or  below  an  action  level 


ITRC,  ISM-1,  Section  4.2.1 


7  1(b).  How  “badly”  might  I 
underestimate  the  mean? 
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CV 

Frequency 

Magnitude 

True  Mean 

Estimate 

1 

1  out  of  4 

15% 

400  ppm 

<  360  ppm 

2 

1  out  of  4 

25% 

400  ppm 

<  320  ppm 

3 

1  out  of  4 

30  -  60% 

400  ppm 

160  -  280  ppm 

Coefficient  of  variation  (CV)  =  St  Dev  /  mean 


ITRC,  ISM-1 ,  Section  4.2.1 ,  Figure  4-2 


Can  a  95UCL  be  calculated? 
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Answer: 

Yes,  even  with  as  few  as  3  ISM  samples. 


►  Need  at  least  3  replicates  (r  >  3) 

►  Fewer  methods  are  available  than  we  are 
used  to  with  discrete  sampling: 

•  Student’  s-f 

•  Chebyshev 

►  Each  ISM  result  provides  an  estimate  of  the  mean  (“x-bar”) 


►  Parameter  estimates  are  calculated  directly  from  ISM  data 
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2(b).  How  do  I  choose  a  UCL  method? 
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►  Consider  performance 
measures  (informed  by 
simulation  study) 

•  Coverage 

(probability  UCL  >  mean) 

•  Magnitude  of  difference 
between  UCL  and  mean 


CV  =  30 


0  100  200 


Concentration 


►  Recognize  the  key  to  performance  is  variability 

•  Distribution  of  discretes  ^  Distribution  of  ISM  results 
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Distribution  of  Means  (ISM  Replicates) 
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ISM  (N=30) 
Discrete 


►  ISM  distribution  variance  is  smaller 

►  ISM  distribution  shape  becomes  more  non-normal  with 
increasing  CV  of  discrete  distribution 


ITRC,  ISM-1,  Figure  4-3 
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Coverage  Probabilities 
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Dispersion  Among  Individual  Increments 

UCL  Method 

Low 

(CV  <1.5  or 
GSD  <3) 

Medium 

(1.5  <  CV<  3  or  3  <  GSD 
<  4.5) 

High 

(CV  >3  or 

GSD  >4.5) 

Student’s-t 

Yes 

No 

No 

Chebyshev 

Yes 

Yes 

Maybe 

►  Both  methods  provide  desired  95%  coverage  when  variability 
is  low 

►  Chebyshev  has  more  consistent  95%  coverage  for  medium 
and  high  variability 

►  Increasing  r(> 3)  and  n  (>30)  provides  marginal  improvement 


ITRC,  ISM-1 ,  Table  4-4,  Sections  4.3;  Appendix  A 


12 


How  much  does  Chebyshev 
overestimate? 
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►  Chebyshev  will  tend  to  yield  10-45%  higher  UCLs  than 
Student’  s-t  depending  on  the  CV  of  the  replicates 

►  Example:  Student’ s-t=  100  ppm,  Chebyshev  =110  -145  ppm 


ITRC,  ISM-1,  Section  4. 3. 1.1 


13 


2(c).  Can  I  use  ProllCL  to  calculate 
the  95UCL? 
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Answer: 

No.  However,  there  are  other  tools  available  to 
calculate  a  95UCL  from  ISM  data. 


►  ProllCL  is  designed  to  work  with  discrete  sample  data. 

►  ISM  replicates  are  fundamentally  different  from  discrete 
samples,  and  data  are  typically  available  for  only  a  few 
replicates. 

►  The  ITRC  guidance  document  has  a  link  to  a  calculator  that 
works  for  ISM  data. 


ITRC,  ISM-1,  Sections  4.2.2 
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2(d).  What  does  the  variability  in  ISM 
reveal? 
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Answer: 

High  variability  can  indicate  lab  error.  Low 
variability  does  not  ensure  that  the  results  are 
sufficiently  accurate  to  avoid  decision  error. 


►  RSD  is  the  ratio  of  statistics  calculated  from  ISM 
replicates 

•  RSD  =  SD  /  mean 

►  If  the  goal  is  to  make  sure  that  the  mean  is  not 
underestimated,  a  95UCL  should  be  calculated 
regardless  of  whether  the  RSD  is  high  or  low 


ITRC,  ISM-1,  Section  4. 3. 4.4 
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2(d).  What  does  the  variability  in  ISM 
reveal? 
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Imprecise 


Precise 


Unbiased  Biased 


►  Accuracy  reflects  both  bias  and  precision 
(reproducibility) 


ITRC,  ISM-1 ,  Section  4.3.1 ,  Figure  4-6  and  Appendix  E 
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3.  Is  there  a  preferred  ISM  sampling 
design? 


*  INTERSTATE  * 


*  AMOivnnoay  * 
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3.  Is  there  a  preferred  ISM  sampling 
design? 
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o  o 


Simple  Random 


Systematic 

ITRC,  ISM-1,  Section  4. 3. 4. 2 


Systematic  (3  replicates) 


18 


3.  Is  there  a  preferred  ISM  sampling 
design  (continued)? 


*  INTERSTATE  * 


★  AUO±\nnD3U  * 


Answer: 

Each  random  sampling  design  yields  unbiased 
estimates  of  the  mean  and  is  an  acceptable 
approach  in  most  situations. 


►  Systematic  random  sampling  is  most  often  used 
because  it  is  the  easiest  to  implement 


ITRC,  ISM-1,  Section  4. 3. 4. 2 


100  200 
Concentration  (mg/kg) 
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*  INTERSTATE  * 


3(b).  How  many  increments? 


< 

l 


l 


►  As  the  number  of  increments  increases: 

•  spatial  coverage  improves 

•  variability  between  ISM  results  decreases 

•  95UCL  will  tend  to  be  closer  to  the  mean 

►  Size  of  DU  can  be  a  consideration  -  large  DUs 
may  require  more  increments 


10 


20  30  40  50  60 


70  80  90  ioo 


ITRC,  ISM-1,  Section  4.3.4. 1 
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3(c).  How  many  replicates? 
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Answer: 


r  =3 


►  For  most  DUs,  3  is  sufficient. 

►  Minimum  number  to  calculate 
standard  deviation  (and  95UCL) 
of  ISM  results 


~*3«- 

*  K 


►  More  replicates  will  produce  a  95UCL  closer  to 
the  actual  mean,  but  may  not  be  cost-effective 
unless  the  result  is  near  the  action  level 


ITRC,  ISM-1,  Section  4.3.4. 1 
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4.  Can  I  extrapolate  results  across 
DUs? 


*  INTERSTATE  * 


*  AMO±VnnD3R  * 


►  Unsampled  DU  -  extrapolate  estimate  of  mean 

►  DU  with  1  ISM  -  extrapolate  estimate  of  variability 

•  Standard  deviation  (SD) 

•  Coefficient  of  variation  (CV) 

DU-1  2  DU-2 


ITRC,  ISM-1 ,  Section  4.4.2 
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4(b).  Extrapolation  of  the  Mean 


*  INTERSTATE  * 
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Answer: 

Assumes  the  mean  concentration  in  the  unsampled 
DU(s)  is  the  same  as  the  sampled  DU. 


►  DU-1: 


DU-1  2  DU-2 


ITRC,  ISM-1 ,  Section  4.4.2 


•  Mean  =  100 

•  SD  =  ? 

DU-2: 

•  Assumed  Mean  =  100 
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4(c).  Extrapolation  of  the  Variance 


*  INTERSTATE  * 
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Answer: 


Assumes  that  the  variability  is  similar  in  all  DUs. 


DU-1  =  DU-2 


ITRC,  ISM-1 ,  Section  4.4.2 


►  DU-1: 

•  Mean  =  100 

•  SD  =  50 

•  CV  =  50/100  =  0.5 

►  DU-2: 

•  Mean  =  400 

•  Assumed  CV  =  0.5 
/  400,  therefore 

SD  =  200 


=  SD 
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5.  Can  background  and  site  ISM  data 
be  compared? 
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Answer: 

Yes,  but  statistical  tools  for  comparison  are  limited. 


Background 


DU-1 

f(x)  [ 


0  100  200 

Concentration  (mg/kg) 


►  Is  the  site  mean  equal  to 
background? 

►  Hypothesis  testing  is  limited 

►  Cannot  test  upper  tails  with  ISM 
data 


ITRC,  ISM-1,  Section  4.4. 3. 3 
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5.  Example  Background  Comparison 
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Reference  Area 

(mean  =  0.17) 

ITRC,  ISM-1,  Section  7.2.4,  Figure  7-1 


Site 

(mean  =  0.18) 
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6.  Can  ISM  and  discrete  or  composite 
data  be  compared? 
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Answer: 

Yes,  if  the  sample  support  is  known. 


►  Measurement  variability  is  a  function  of  sample 
support 


►  Comparisons  are  possible  with  caution  and 
consideration  of  all  differences  in  sampling  and 
analysis  methods  and  meaning  of  data 

•  Sample  collection  •  Analysis  method 

•  Sample  prep  •  ISM  estimates  mean 


ITRC,  ISM-1,  Section  4.4. 3. 3 
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*  INTERSTATE  * 


Summary:  Statistical  Design 


u 

z 

3 

0 

u 


►  Mean  or  95UCL  from  ISM  data  may  be  used  to  make  decisions 

►  3  replicate  samples  provide  adequate  information  for  a  95UCL 

►  Systematic  random  sampling  is  most  commonly  used 

►  About  30  increments  per  ISM  sample  is  usually  sufficient 

►  Extrapolation  of  the  mean  or  variance  can  be  very  uncertain 

►  Comparisons  between  ISM  site  and  background  data  are 
possible,  with  caution 

►  Comparisons  between  ISM  and  other  data  are  possible,  with 


caution 
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